A Simple Algorithm for Maximum Margin Classification, Revisited

Let P be a point set of n points in R. Every point has a label/color (say black or white), but we do not know the labels. In particular, let B and W be the set of black and white points in P . Furthermore, let ∆ = diam(P ), and assume that there exist two parallel hyperplanes h, h′ in distance γ from each other, such that the slab between h and h′ does not contain an point of P , and the points of B are on one side of this slab, and the points of W are on the other side. The quantity γ is the margin of P . A somewhat more convenient way to handle such slabs, is to consider two points b and w in R. Let slab(b,w) be the region of points in R, such that their projection onto the line spanned by b and w is contained in the open segment bw. We use (1 − ε)slab(b,w) to denote the slab formed from slab(b,w) by shrinking it by a factor of (1− ε) around its middle hyperplane. Formally, it is defined as (1− ε)slab(b,w) = slab(b′,w′), where b′ = (1− ε/2)b + (ε/2)w and w′ = (ε/2)b + (1− ε/2)w. In the following, we assume have an access to a labeling oracle that can return the label of a specific query point. Similarly, we assume access to a counterexample oracle , such that given a slab that does not contain any points of P in its interior, and supposedly separates the points of P into B and W, it returns a point that is mislabeled by this classifier (i.e., slab) if such a point exists. Conceptually, asking queries from the oracles is quite expensive, and the algorithm tries to minimize the number of such queries.